{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Linear Regression: Inverse Matrix Method\n",
    "\n",
    "This script explores how to accomplish linear regression with TensorFlow using the matrix inverse.\n",
    "\n",
    "Given the system $ A \\cdot x = y $, the matrix inverse way of linear regression (equations for overdetermined systems) is given by solving for x as follows.\n",
    "\n",
    "$$x = \\left( A^{T} \\cdot A \\right)^{-1} \\cdot A^{T} \\cdot y$$\n",
    "\n",
    "As a reminder, here, $x$ is our parameter matrix (vector of length $F+1$, where $F$ is the number of features). Here, $A$, our design matrix takes the form\n",
    "\n",
    "$$\n",
    "A=\n",
    "\\begin{bmatrix}\n",
    "    1 & x_{11} & x_{12} & \\dots  & x_{1F} \\\\\n",
    "    1 & x_{21} & x_{22} & \\dots  & x_{2F} \\\\\n",
    "    \\vdots & \\vdots & \\vdots & \\ddots & \\vdots \\\\\n",
    "    1 & x_{n1} & x_{n2} & \\dots  & x_{nF}\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "Where $F$ is the number of independent features, and $n$ is the number of points.  For an overdetermined system, $n>F$. Remember that one observed point in our system will have length $F+1$ and the $i^{th}$ point will look like\n",
    "\n",
    "$$point_{i} = \\left( y_{i}, x_{i1}, x_{i2}, \\dots, x_{iF} \\right)$$\n",
    "\n",
    "For this recipe, we will consider only a 2-dimensional system ($F=1$), so that we can plot the results at the end.\n",
    "\n",
    "We start by loading the necessary libraries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "from tensorflow.python.framework import ops\n",
    "ops.reset_default_graph()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we start a graph session."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "sess = tf.Session()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For illustration purposes, we randomly generate data to fit.\n",
    "\n",
    "The x-values will be a sequence of 100 evenly spaced values between 0 and 100.\n",
    "\n",
    "The y-values will fit to the line: $y=x$, but we will add normally distributed error according to $N(0,1)$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Create the data\n",
    "x_vals = np.linspace(0, 10, 100)\n",
    "y_vals = x_vals + np.random.normal(0, 1, 100)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We create the design matrix, $A$, which will be a column of ones and the x-values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Create design matrix\n",
    "x_vals_column = np.transpose(np.matrix(x_vals))\n",
    "ones_column = np.transpose(np.matrix(np.repeat(1, 100)))\n",
    "A = np.column_stack((x_vals_column, ones_column))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now create the y-values as a matrix with Numpy.\n",
    "\n",
    "After we have the y-values and the design matrix, we create tensors from them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Format the y matrix\n",
    "y = np.transpose(np.matrix(y_vals))\n",
    "\n",
    "# Create tensors\n",
    "A_tensor = tf.constant(A)\n",
    "y_tensor = tf.constant(y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we solve for the parameter matrix with TensorFlow operations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Matrix inverse solution\n",
    "tA_A = tf.matmul(tf.transpose(A_tensor), A_tensor)\n",
    "tA_A_inv = tf.matrix_inverse(tA_A)\n",
    "product = tf.matmul(tA_A_inv, tf.transpose(A_tensor))\n",
    "solution = tf.matmul(product, y_tensor)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run the solutions and extract the slope and intercept from the parameter matrix."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "solution_eval = sess.run(solution)\n",
    "\n",
    "# Extract coefficients\n",
    "slope = solution_eval[0][0]\n",
    "y_intercept = solution_eval[1][0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we print the solution we found and create a best fit line."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "slope: 1.00690733023\n",
      "y_intercept: -0.119504081442\n"
     ]
    }
   ],
   "source": [
    "print('slope: ' + str(slope))\n",
    "print('y_intercept: ' + str(y_intercept))\n",
    "\n",
    "# Get best fit line\n",
    "best_fit = []\n",
    "for i in x_vals:\n",
    "  best_fit.append(slope*i+y_intercept)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We use Matplotlib to plot the results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAEACAYAAACnJV25AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl8FFW2wPHfzQISElYBASEJcYdxR1RUEnFcRlHkuSc4\nCKMMKsomBDQmmAFRUUdx1IeiiOLTER1RdFwQgjoO4sYgiKBJCAgKyBZABkg4749KoDu9pLu6eknn\nfD+f/gzdVV11u+Ocun3r3HONiKCUUip+JES7AUoppZylgV0ppeKMBnallIozGtiVUirOaGBXSqk4\no4FdKaXiTMCB3Rgzwxiz0RizzOW1B40xK40xS40xrxtjWoSnmUoppQIVTI/9eeCiOq99AHQXkZOB\nH4DxTjVMKaWUPQEHdhH5FNhW57X5InKg5uli4EgH26aUUsoGJ8fYBwP/dPB4SimlbHAksBtj7gb2\ni8jLThxPKaWUfUmhHsAY80fgD8D59eynRWmUUsoGETHB7B9sj93UPKwnxlwMjAUuF5G9ATROHyIU\nFhZGvQ2x8tDvQr8L/S78P+wIJt3xZeAz4BhjzFpjzE3ANCAV+NAY87Ux5klbrVBKKeWYgIdiROQG\nLy8/72BblFJKOUBnnkZBdnZ2tJsQM/S7OES/i0P0uwiNsTuGE/SJjJFInUsppRqa8vIKCgpmsn79\nATp3TqC4eBCZmekYY5Agb56GnBUTqoyMDCoqKqLdDGVTeno6a9asiXYzlGrQyssr+P3vp1FaOhFo\nDuxm8eJCPvxwuK3jRT2wV1RU2L7zq6LPmKA6EkopLwoKZroEdYDmlJZOpKBgqq3j6Ri7UkpF2fr1\nBzgU1Gs1Z8OGA952r5cGdqWUirLOnROA3XVe3U2nTvZCtAZ2pZSKsuLiQWRlFXIouO8mK6uQ4uJB\nto4X9ayYmju+EWmDcp7+/ZTy5CvDJZD3bNhwgE6dQsuK0cDuR0ZGBps2bSI5OZnExEROOOEEBg4c\nyC233FLvTcOKigoyMzOpqqoiISF+fxjF8t9PqWjwluGSlWVluNQX3L2xE9hjNuKUl1eQlzeRnJxC\n8vImUl4efEpkqMcwxvDOO++wY8cOKioqyM/P54EHHmDIkCH1vldENOgp1Qj5znCZGblGRLCQjXjj\n7fWysjWSlTVaYJeACOySrKzRUla2xusxvHHiGBkZGfLRRx+5vbZkyRJJSEiQFStWyDvvvCOnnHKK\ntGjRQrp27SpFRUUH9+vataskJCRIamqqpKWlyeLFi6W0tFTOP/98adu2rbRr105yc3Nlx44dAbcn\nFvn6uyrVWGVn31sTc9wfOTn32jpezf/Hgoq3Mdljd+KKF66rZs+ePTnyyCP55JNPSE1N5cUXX2TH\njh288847PP3007z11lsAfPzxxwBUVlZSWVlJr169EBEmTJjAL7/8wsqVK/npp58oKioKqT1Kqdji\ndIaLHTEZ2J3I6XQ6L9RVp06d2Lp1K+eddx7du3cHoEePHlx33XUsWrTIbV9xGYrJysqib9++JCUl\n0bZtW0aOHOmxv1KqYXM6w8WOqM889ebQFc81MAd3xXPiGL6sX7+eNm3asGTJEvLz81m+fDn79u1j\n3759XH311T7ft3nzZu644w4++eQTdu3aRXV1NW3atAm5PUqp2JGZmc6HHw6noGCqS4aLvRuntgU7\ndmP3QRyNsScmJsry5cslKytLHnvsMdm3b5+IiIwYMUIGDhwoIiIVFRWSkJAg1dXVB987ZMgQueGG\nG2Tbtm0iIvLmm29Kly5dAm5PLPL1d1VKOQMbY+wx2WN34orn9FVz586dLFq0iBEjRjBw4EC6d+/O\nrl27aN26NcnJySxZsoSXX36Ziy66CIB27dqRkJBAaWkpRx999MFjtGrVihYtWrB+/XoeeughW21R\nSil/NI/dj8zMTDZt2kRSUhIJCQkH89iHDh2KMYY33niDUaNGsW3bNvr06UNGRgbbt29n1qxZABQV\nFfHkk09SVVXFe++9R2pqKjfeeCOrV6/mqKOOYuDAgTz66KOsXbs2yp/Uvlj++ykVD3SCkoo4/fsp\nFV5xNUFJKaWUPRrYlVIqzmhgV0qpOKOBXSml4owGdqWUijMBB3ZjzAxjzEZjzDKX11obYz4wxqwy\nxrxvjGkZnmYqpZQKVDA99ueBi+q8lg/MF5FjgQXAeKcappRSyp6AA7uIfApsq/PyFcALNf9+Aejv\nULuUUkrZFOoYe3sR2QggIr8A7UJvkgJ46qmnOOKII2jRogVbt24lLS2NNWvWBPz+hIQEysrKABg2\nbBiTJk0KU0uVUk4sDORBBBYutPXWoGaeGmPSgbdF5MSa51tFpI3L9i0i0tbHe6WwsPDg8+zsbLKz\ns2N65mLt0nhJSUkkJydz9tln8/TTT9O5c+eQjpuZmcmMGTM4//zzvW6vqqqiRYsWLFmyhB49enhs\nv+mmm+jSpQv33Xefz3MkJibyww8/0K1bt5DaWp9Y/vupxs3OuqN2z+PkUnglCxdS8swz8PHHsH49\nEyHomafBVmhMB5a5PF8JdKj59xHASj/v9Ve5LCZlZGTIggULRERk7969MnjwYLnyyisdOW7dqpGu\n1q1bJwkJCVJVVeV1+6BBg6SgoMDvOYwxUlpaGlI7AxHLfz/VeDlR3TVQublFLueRg+fLzS3yun9Z\n2RrJzS2S7Ox7JTe36FCbqqpEXnlF5MQT3ZZewkZ1x2ADewbwrcvzB4BxNf8eB0zx816vHzKWA0Pd\nAPzuu+/Ksccee/D53r17ZfTo0dK1a1c54ogjZNiwYfLf//5XRER+/fVXueyyy6RVq1bSpk0bOe+8\n80REZODAgZKQkCApKSmSlpYmDz30kNs5V69eLc2bN5eEhARJS0uTvn37isihQD19+nRJTk6Wpk2b\nSlpamlx++eVe2+4a2F0vBCUlJXLkkUfKww8/LO3bt5dOnTrJ888/H9Bn8iaW/36q8Qo22IYimKXw\nPC8430mr5pdKcbdLZENaW8+DNGkS3qXxjDEvA58Bxxhj1hpjbgKmAL83xqwCLqh57hxjnH2E4Lff\nfuPVV1/lrLPOOvja2LFj+fHHH1m2bBk//vgj69evPzg88vDDD9OlSxe2bNnCpk2bmDx5MgCzZs2i\na9euzJs3j8rKSsaMGeN2nqOPPpoVK1YAsGPHDubPn1/zVVjtv/nmm8nNzWXs2LFUVlYyd+7coD/L\nL7/8ws6dO9mwYQPPPvsst912Gzt27Kj3MynVUIRzBbW6glkKz3XJzsNYxW3cztLdy7in7J903Lnl\n0I4pKTBqFNTcJwtWwPXYReQGH5susHXmBqJ///4kJSWxc+dOOnTowPvvv39w27PPPsu3335Ly5ZW\n+n5+fj65ublMmjSJ5ORkfv75Z8rLy8nKyqJ3795ux5UAxqVF5GBAD2T/QDVp0oSCggISEhK45JJL\nSE1NZdWqVZxxxhl+P5NSDUU4V1Crq7h4EIsXF3qMsRcXD/fYd/36A6RRzZ95kFHcxxF1LgjbaUlJ\nj5Pov/B1OPxw222KyYU2YsncuXPJyclBRHjzzTc577zzWLlyJcYYfvvtN0477bSD+x44cOBgAL7r\nrrsoKiriwgsvxBjDzTffzLhx46L1Mdy0bduWhIRD/4GnpKSwa9cuNm/e7PczKdVQBBNsQxXwoj5b\ntjB8yyJyeJzWbHfbtIl2PMIonuRWTm/3MP1DCOoQ64E9BgJKbVAzxnDllVcydOhQPv30U6688kpS\nUlJYsWIFHTt29HhfamoqU6dOZerUqaxcuZLs7GzOOOMMcnJyDvbC7Qr1/b4cfvjhfj+TUg1FpNcd\nzcxM56WXCr1v/PlnePhhePppBux276GvozMPMZZn+RN7SMGpXxVaKyYIc+fOZfv27ZxwwgkHe+Ej\nRoxg8+bNgLXI9QcffADAO++8Q2lpKWAF+aSkJJKSrOtohw4dDuaY++KvlxzI++2o7zMp1ZDUBtsF\nCyby0kuFkV1MGqC8HIYNg8xMK7C7BPVf0towuVtfTmrel2kMORjUrV8Vg0I+tQb2evTr148WLVrQ\nsmVLCgoKmDVrFscddxwADzzwAEcddRRnnnkmrVq14sILL2T16tUA/PDDD1xwwQWkpaXRu3dvbrvt\nNs4991wAxo8fT3FxMW3atOGRRx7xet66vXLX50OGDGHFihW0adOGAQMGBPR+f1z3nTJlis/PpFRj\nY2vi0fffwx//CEcfDU8/DXv3HtrWowe8/DJHbN3IhNL5fPXtfeTmTiUnp5Dc3Km2c9/r0qXxVEj0\n76fiVdATj775BiZPhtdf9xxGPuMMmDAB+vWDhOD607o0nlJKOcQ1NdHSnNLSiRQUzHTf8bPP4NJL\n4dRTYc4c96CenQ0ffgiLF8MVVwQd1O2K7ZunSinlkGBLDPjNhReB+fNh0iRYtMjzzZdeCnffDS7z\nXiJJA7tSKu55DqusZO7c4fTocTxZWSleg7y3XHjDTi6rXg29esEXX7ifxBi46ipryOXkk8P8ifzT\nMXYVEv37qYYgL28is2ePwQrSFcA0wP/YuevFIJGmXMOLFDYZy7H7fnU/eFIS5OVBfj4ce6zHuUMt\nRmZnjD2o+gOhPGiAtWJU/fTvpxoC93ougdeRKVu5Sp7p1U9+atbas45L06Yit90mssZ3YTEnipFh\no1aMDsUopWJeqL1e92EV/3Vkyssr+Mv46Zzy5ZfcsOHf/GnPTrc99yQ14c2Op/P+Cd3Ztq4ZlYOe\n89km3zdgp/qe0OSEYK8Edh/46Nmlp6cLNaUp9dHwHunp6QH3PJSyw4ler/sxfPfYy5cuk6ltzpFN\neFZarGrZUv7a+ixpzVqBNQIj621TMJUffcFGjz3qgV0ppfxxqgRvbR30M8+8U1JTb3ILyj0zhsm2\n226XXclNPaLwz7SXl0/5vQy5ZrzLewJrkxNttxPYdShGKRXTnCrB61rPxRramcq+8u3ctHUxF1X8\nh4S/7XHbv4KuPMhYnmMwZ7WagmzCpR2BtSmSxchcaWBXSsW0cJTgzTxQxUvNN8CXM2HfPrdtqzia\nKYznJfKoIrnOuWrbEVibIl2M7KBgu/h2H+hQjFLKBkeXuVu+XCQ3VyQhwXPg+6STZOO0J+Tobt7H\nzt3bEdgYuxOwMRQT9Tx2pZSqT21WzKFeb5ALU3/5pTVL9M03PbeddZY1S/QPfwBj/J7LdVuLFpWI\nJLFzZ4q9NgXITh67BnalVIPmMxVSBD75xAro3kpPX3CBFdD79Al56cxw0sCulGpUvFZg7HYvn97z\nO46Y8Sz861+eb7r8ciugn3FGpJtriwZ2pVSj4loqwHCAAbzBBP7CqfzHfceEBLj2Whg/Hn73u6i0\n1S47gV2zYpRSYRXorFE7s0vXrz9AEk24nlmM536O53v3HZKT4cYbYdw4a+GLENvYYAR7t9XuA82K\nUapBqJ3Ik519r+TmFoWU6RFoRoutzJc9e+S5npdKGekeGS7/TUwSueMOkbVrHWtjtKAzT5VSoXA6\nyAU68zKoGZo7d4pMnSrSsaNHQN9OC3my1RmyZsmXjrcxWuwEdkeW8zDGjDTGLDfGLDPGzDbGNHHi\nuEqpyAp41aAABTprNKD9tm2D++6D9HQYMwZ+/vngpp1Nm/FcZjZ3XT2Mi7/+O+k9T3O8jQ1JyGPs\nxphOwHDgOBHZZ4x5FbgOmBXqsZVSkeV0kAt01qjf/TZuhL/+Ff72N9jpXmmRTp1gzBjSbrmFwc2b\nMziMbWxInGp5ItDcGJMEpAAbHDquUiqCDgU5V/aDXHHxILKyCl2OWVsrZVC9+53TdSR/SyyHjAyY\nMsU9qGdmwtNPQ1kZjBwJzetejJxvY0PiSLqjMeYOYBLwG/CBiAz0so84cS6lVPh4zQv3srpQsMcM\nZNZo7X7mx80M+fUz+qxdjtm/332n44+3lp677jpr5SKHhDyzNYyiksdujGkFvA5cDewA5gCvicjL\ndfaTwsLCg8+zs7PJzs4O6dxKKedFLcgtWwaTJ8Nrr8GBOkM/p55qTSrq39/KSY9jJSUllJSUHHw+\nceLEqAT2q4CLROTmmucDgV4icnud/bTHrlQD45rf3bKlVRulsjLF2VzvxYutgP72257bzj3X6qFf\ndFFMT/sPh9rvfvbsoqADuxNpjGcA3wKHAQaYCdzmZT/n84CUUmET1mqGBw6IfPSRyPnne6QsCohc\nfLHIxx87+4EaEPfvPgrpjiKyBGv45RvgPzXBfXqox1VKHVJeXkFe3kRycgrJy5tIeXlF2M/pnvo4\nEygm5DRIEZg3D84+G/r2hQUL3LcPGGBVYvznP63eepCi8T2Fg2faaXAcufsgIhOBiU4cSynlztsN\nzcWLQ7uhGQj31McQ0yCrq2HOHGvIZdky922JiXD99VYdlxNOsN3eaH1P4eA97TRw8X0XQqk44PSk\noUC5pz4GlwZZ23P+fZ97mH5Wf/YdfYyVyeIa1Js0gaFDYfVqePHFkII6RO97CgfvaaeB08CuVIyL\n1sxI9/zuQUABgeR6l5dXcFnfR2g1O5UZH8/ilsVzaVJedmiHlBQr97yszMpF79bNkfbG0wxSz9z6\n4Gh1R6ViXDAzI52sUlh3vc4WLQSRIpcVg7wMcVRW8vmAP7Gg/D90YLPbpt3JTWk+7i648044/HBb\nbfInnmaQun73s2fbOECwd1vtPtCsGBWjnKxmGA5hrZDolF9/Fbn3XpFWrTwyXDbSTvKZLJedOzas\nTYj1Ko12odUdlQpOQwkGtRefnBzfF5+oVCncsEFkzBiR5s09AvpajpThPCbN2B2xaomBfE8NjQZ2\npYIU6yVbg5Gdfa/XlPCcnHudP1l5uciwYSJNm3qccF96huS3u1CS2RrTF8uGwk5g1zF21ajF0w23\niIwxr1oF998Ps2dDVZX7th49YMIEkq++mlvWrWddweMuZQkaXsphQ6aBXTVq8XTDrbh4EIsXF3oU\n8CouHh76wb/5xspBf/11qxPuqmdPq45Lv34H67hkZqbz0kuFXg6kIkEXs1aNWjiqGUaT4wW8PvsM\nJk2Cd9/13JadbQX0vn0bXR2XSIpKdceAT6SBXcWoWC7ZGi5+0yJFYP58K6AvWuT55ksvtQpznX12\nRNvcWGlgV0rVy+evlPdvI/Pb/1hj6EuWuL/JGLj6amva/8knR6PZjZadwK5j7Eo1MnWn3ifSlDNK\nu5N0Wi/Y4T6piKQkyMuD/Hw49li3TU5OhgpGtM7bkGhgV6qRqc0EasJe/sgLjOMBsiizlsmp1bQp\nDBkCd91lLU1XR7QKbsVToa9wani3/pVSIcnsUMUdPEgpWUxnqBXUa6WmwpgxUF5uLR7tJahD9Apu\nxVOhr3DSHrtSjcWOHfDEEzzz4dMkstVt0/aEw5Dbh9K68F5o06beQ0Ur/z+e5h2EkwZ2pcIs6mPC\nmzfDY4/BE0/Ajh0kumza2qQ5759wFmfN+isZv+se8CGjlf8fT/MOwirYqap2H2hJAdUIRbUWzU8/\niYwYIZKS4llnoGtXkSeeEPntN1uH9vxc30lqaj8588yxYa3R0lBq+zgJGyUFNN1RqTDKy5vI7Nlj\nqNvDzM2dGr6ZmWVl8MADMHMm7Nvnvu2YY6wMl7w8SE4O6TS1v0RKS7exfHklu3ZNIxKTvBrbvAM7\n6Y7aY1cqjOwW5rJVSnjFCpG8PJHERM8TnnSSyKuvilRVOfTJDomnQmqxCC0CplRssTMmHHRK31df\nWXVc3njDc9tZZ1mzRC+9NGzT/vWGZuzROw5K2VS7rmdOTiF5eRMpL6/w2MdziTPfS8rVCjil7+OP\n+e288+D00z2Det++sGAB/OtfcNllYa3l4n19Tr2hGU3aY1fKBn+9asAtC+a5565k+vSpAZew9dsD\nFoH33rN66J9+SkqdveanZHHCS1PpdGV/Bz+tf6FUlYx6xlCc0sCulA2+etUjR97D8uWJIc2M9DZ8\nY9jJFVXfw2mnWSV0XVSTwN+5hslMYPlv3ch9fSovRTCw110b1d/FyzWQt2xZyTffHGDt2snoLFKH\nBTso7+0BtAReA1YCK4BeXvYJ9z0GpSLG103RDh2uDPlGomtKXxL7JI9n5IfkNh4n22cS5BmGyFGs\nDv+KSQ7wTFW8x+d3Fevr0EYSUbx5+hjwrohcbYxJAo9fiEo1SL6GCnzdFBVJJdQbiZmZ6cyfdzML\n/ngdlyz7lI7/3Q77XXZo1gz+9CdGr0ti2pvFHm2I1bFtz185CXj7rkpLt2k9mFAFeyWo+wDSgNIA\n9gvnRU2poATSI/Q3GcbXtssvHxNaj33nTpGHHxbp2NHz50Bamkh+vsjGjfW2z+5nDifPXzne0yQz\nMgZo+qQLorGYNXAS8DnwPPA1MB1o5mW/sH8BSgUi0IBYX352baDMyTkUKG3PjNy6VeS++0TatvUM\n6G3aWNu2bnU7b3b2vXLFFSPk8svHuLUhlM8cTp7f5xqBkR5tOvPMsbZy/+NVtAL7aVg/FE+vef5X\nYKKX/aSwsPDgY+HChWH+OpTyLtAJNaFOLqov2IqI1fvOz7d643VP1LGjyNSpVi/e5dh2AnQsTCLy\n1vYuXW72uDDFQlujaeHChW6xMlqBvQNQ5vL8HOBtL/uF+etQKjCBBuywBpi1a0WGDxc57DDPhmRk\niDz1lMiePR5vs9smuxcppwVy0YuFXxexxE5gD/nmqYhsNMasM8YcIyKrgb7Ad6EeV6lwCXQ2aCj5\n2b6sW1BC6S2j6F32H5Klzg3V44+3lp677jqfdVzszvKMlaqImZnp9dbICSZ9UvkQ7JXA2wNrnP0L\nYCnwBtDSyz7hvawpFaBgeoRBDav4s2yZ7Lysn1RhPLrN/+3RQ2TOHJHq6noPY7fHrr3ghgsbPXat\n7qgapYhVCPz8c2uW6FtveWz6hJ5M4ii+aZfI7y88KqA2+FyIOoBUwMZWFTFe2KnuqIFdxZWYmKIu\nAosWwV/+Ah995LH5PS5iMoP5hCWABmjlnwZ21aiF0pt1PYbtC4MIvPsuTJoE//63x+YvuhzHsHXT\n+YpzsQJ6hOu0qwZJ67GrRi3ULBbb49BVVVat85NP9kw7SUy0aqQvX17n+LGRpaJiHzbG2GNz7rFS\nNoRaFzzgcrm19u+3Vinq3h2uvRaWLj20rUkTuOUWWL0aXnwRunc/mO2RmzuVDh2+xbPU7UrKy5f7\nLQMcjEDKCqv4pNUdVdwINaUv4AvDnj3w3HPw4IOwdq37tpQUGDoURo+Gzp09zlGb7mcNG7mmUq4k\nKekB1qyZxZo1oddHCXqxDhVfgu3i232gQzEqzEJN6at3KKeyUuTBB0U6dPAcQ2nZUuSee0Q2bw6q\nvbWplE7XR2nsszfjCdGYoKRUrAh1YouvCUmTRg+EoiJ4/HHYts39Te3awciRcOut0LJl0O2tvVGa\nk1NY01N3ZX95OV2urnHTwK7iSiAzG/291/XCcHzr3Uw6fBetzjsHdu1y23dLSgsYM5K248Zawy8h\ncnpmaKzMNFVREmwX3+4DHYpRDcWaNSK33irStKnHkMsPdJMhPCPJbHV05qbTM0N1pmn8QGeeKhWC\n77+HKVNg9myoqnLbtK5le8bumMJrDKT64A9dZ/POnZ54pBOZ4oNOUFLKB78Tj775Bu6/H+bMsTq3\nrnr2hLvv5vxHv2Lhovs8jpuTU8iCBRPD/wFUo2UnsOsYu4p7vlL/Pp5yNp2en2HNFq2rTx+4+264\n4AIwhk6vLUXHrFVDoT12Fffy8iYye3bt9H2hLx9xN/eRwyeeO//hDzBhAvTu7fayE+UKYlVM1NdR\nPulQjFIuagPWvHmlVO6YST/eZgKT6cUS9x2Ngf/5Hyugn3JKvceLpzHreL5gxQsN7CqsGlLPrjZg\nrSkt4BoGM57v+V2d9V+qjSHxxhshPx+OOy5KLY0u918ztbQYWSzRMXYVNg1tinrRhGfJLs0kn9M4\nilK3bf+lKXNaHEefeU/Q5dxzotTC2KATmeKT3vlRAQm6QFa0/PYbPP44D/3jcZ7ldregvotkpjU9\ngTEDbqf30rmNPqiD60QmV3pTuKHTv54KSKR7dkFXJtyxw0pZzMiAO++k/d7Kg5u20poiCknnRz6/\n6hqeeH1qTP7KiIbi4kFkZRVyKLjXrus6KGptUqHToRjlxtc4eiSnqAc17PPrr/DYYzBtmhXcXWxO\nTOGh6vE8xZ3sIiHkhajra3NDuf/gSheOjlPBTlW1+0BLCsSM2qqC2dnuCzT7m4YeySnqAVUmXL9e\nZORIkZQUz0qLXbqITJsm5d9978xC1PXQ6fsqnLBRUkADeyPjLwjVF1Bdy8yGM1BmZ/tZXai0VGTo\nUJEmTTx3OOYYkeeeE9m7Nyzt8kVL5KpwshPYdSimkfF9E3RqvePooVRODIa3YZ/j+ZL7yt+EYyZB\ndbX7G046CcaPh6uugsTEsLevLs0sUbFGb542Mv6CUKxkSLje0DuVr3id3/MdPTlnzTL3oH7mmfD2\n21atl2uvjUpQB80sUbHHsf/yjDEJxpivjTFvOXVM5Tx/QShWMiQyM9P5eFIv/tPxJL7idAYw3237\nv5p15efZL8Nnn8Fll1kzR6MoVr43pWo5NvPUGDMSOA1oISKXe9kuTp1L2VffFPKoTpsXgfffh0mT\n4NNPPTa/RT8mcTdL6BFzMyPjsdyAig1RKylgjDkSeB6YBIzSwB7bYi4IHTgAb77J3sIimi7/1m1T\nNYa/cy33M55vOfHg67XlchtqmqFSgYpmSYFHgbuA4BZ9VFERqZug9aqqgv/7P2ti0cqVNHXZtJ8k\n3kw7jo969uJ/FzyGt/z5hlbmQKlICTmwG2MuBTaKyFJjTDbg88pSVFR08N/Z2dlkZ2eHenrVwJSX\nV3DfhGf43ddLydvwL9rv2u62fQ+H8Sx/4iHuYt3OtlyRdg9ZWZ4LTBcXD/eb4RMTFy6lbCgpKaGk\npCSkY4Q8FGOMmQzkAVVAMyANeENEbqyznw7FRFisDVOsWbGSl/sM449bVtOZn9227U5swrTqkTzK\nSDbR4eDrOTmFzJgx2OvQUU5OISUlnqsX6apGKp5EZShGRCYAE2oa0AcYXTeoq8jzHKZYydy5w+nR\n43iyslL8pdFDAAASxElEQVQiG+S3b4cnnqDtXyYzYe8et01baMPCE0/kg2N68cycArwNufgaOopk\nmQOlGpRgZzT5ewB9gLd8bHNwLpaqj/tsyDUCzk5591WWwM3GjSL5+SJpaR6zRNfTUUYxVZqzU3Jy\n7rU1LV+n8qvGAC0poGq5T8t3dsp7vQF13TqRO+4QadbMI6CXkS5DeUqasseRkgWRKnOgVLTYCexa\nUiBOuQ9TODvl3ddNyydGjOfh9nvghRdg/373Nx13HJv/dDOXPLmOVWUDgcNwvREK9rJ1YibDR6kY\nooE9ThUXD2Lx4tpsEmfHouuWJejBt4znfq596xWgzg3yU06Bu++GK6+kXUIC/xxQoSVilQozXfO0\ngfOX+VK7rbR0G8uXV7Jr1zScWLC4dp3MnqzgbiZxBV6qSJxzjrU49MUXR33Kv1INmS5mHad8Be9g\nVph3bLapCD+/8iqlQ8Zxzp61ntsvvNDqoZ93nu3PpZQ6xE5g15unMS6U+umOOnBAZN48kbPP9rgh\nKiC7LrxI5IsvHPlcgby33owcpeIEmhUTf/wFb78LUgTBb6CsqhL5+99FTj7Z80SJiSK5uSLLlzv6\nueprq6Y4qsbETmDXm6cRFuzwQ2D10+3fFPVZb+XdP5P52acwZQqsWuX+piZNYNAgGDcOunXz+7l8\nvW53cQotI6BU/TSwR5CdolX+grd75ot7HZVA1Q2Uh5HARaWdSDvldPjNfXFomjWDoUNhzBjo3Lne\nz/Xcc1cyePA/vH5euxclXa1IqQAE28W3+0CHYrwMP6wRuEfatx/oc6y4vqGHUCfo1A7npFIpY3hQ\nfqaD55BLixYiEyaIbNoU4Oey2pmRMcDncIvdIRVdX1Q1NugYe2xzHxMPfJp/OGdX3nLVWClkvGyh\ntWdAP/xwkUmTRLZvD+JzHXq0ajXQ7z0AuzNNdYxdNSZ2ArsOxUSQ+/DDTCCwseKwzK785Rd45BGe\n+ueTJNRZKu+XxFSSx4+kbf44aF532MOTr2GVVq12s3277+EWuzNNP/xwuE5yUsqfYK8Edh9oj71O\nb9OZjJagrVkjcuutIk2bepx8/WGt5Zle/aRs5aqgDumrF71o0afau1YqRNjosesEpQirzRKZP/8/\nbNz4InV7s2Fby3PVKivD5aWXrJWLXHXvbs0SveYaSLL3I87XBKiYW4ZPqQZGZ542IMHMGq37vqBm\nay5dCpMnw5w5VqfZVc+e1izRfv0gQWuYKxWLNLA3MMH2ZoO6GPz731ZAnzfP80B9+lgB/YILtI6L\nUjFOA3ucqy2+5XP4RgQ++ggmTQJvayZecokV0Hv3jlCLlVKhisrSeCpyfE3O+Xl9Nbz1lhXQlyxx\n32wMDBhgjaGfemqkmqqUiiIN7A1I3bTCBKq5mhd5cOn/whXF7jsnJkJenjXt//jjI95WpVT06FCM\nTYHUQXe6HG3tGPva0rsZyBvkcz9HU+q+U9OmMHgwjB0LGRkhn1MpFV06FBMh/mq+AEHXgwlUZod2\nfJ6bClMzaPtbpfvG5s1h2DAYNQo6dgzpPEqphk177DZ43sSsAJ6lffsKUlJ2s2bNLILNT/fby6+s\nhCefhEcegc2b3d/YujXccQcMHw5t2zr2GUOhC2go5RztsUeI+03MCmAaMJFNm5oD9xBs9UFfvwA+\nejWX9Ln/gGnTYPt29ze1bw+jR1u99LQ0Zz6Yj7YFE6TtVLBUSjlLA7sN/mu+JBNsOdq6pXM7soNh\npfvo0KsXVO9337lLF2v8fMgQq4xuGNkJ0lovXanoC3m6oTHmSGPMAmPMd8aYb40xdzjRsFhUXl5B\nXt5EfvxxG6mpw7ECeN0UxEFAYc02OFQjfZDP49b+AsignKf4M+VkMpppHOYa1I8+GmbMgB9/hNtv\nD3tQB39BeqbP92i9dKWiz4keexUwSkSWGmNSga+MMR+IyPcOHDtmePZeV5Kaej1Nmxq2bHHtoacD\nQ8jIuJHMzB4BVR/smforN3EDN/B3kqh233jiiVYO+lVXWSmMEWQnSDuxqpNSKkTBVg2r7wG8CfT1\n8roDdc6ix9cCD1dcMcJ+BcMvvxQZMEAOGONRafGbph3l52eetRaRjhI7i1povXSlnEW067EbYzKA\nk4HPnTxuLPDVe62sbMGHHw4Orj74J59Ys0Tffx8A19vdX7fK4L3TzuP66RM5oluGw58iOHaW3tN6\n6UpFn2OBvWYYZg5wp4js8rZPUVHRwX9nZ2eTnZ3t1OnDzt8QQ0ALRojABx9YAf2TTzy39+sHEyZw\n6plnEisT/+0G6bAsDKJUI1FSUkKJt1pPQXAkj90YkwTMA/4pIo/52EecOFe02C2zy4EDMHeuFdC/\n+sp9W0KCVQM9Px9OOsmRNmr+uFLxJWrVHY0xs4BfRWSUn30adGCHIMvsVlXBK6/A/ffDd9+5b0tK\nghtvtAL60Uc71jZbFx6lVEyLSmA3xvQGPga+BaTmMUFE3quzX4MP7AHZuxdeeAEeeADKyty3HXYY\n3HwzjBkDXbs6etp6S/oqpRqkqMw8FZF/AZHNwwtRWIYsdu+G6dNh6lTYsMF9W1oa3HorjBwJHTqE\ndh4fNH9cKVWr0c08dXzK+/bt8MQT8Ne/wpYt7tvatIE777TquLRu7UTzfdL8caXUQcHmR9p9ECN5\n7HZys73auFEkP18kLc0jB12OOEJk6lSRnTvD8yG80PxxpeIT0c5jbwhCHrL46Sd46CF45hnYs8d9\nW3q6tbDFTTdZ4+kRpPnjSqlajS6w2x6y+PFH64boCy/A/jqFuY47DsaPh+uvh+Rkh1scOM0fV0pB\nI6zHHnRa4PLlMHkyvPqqlZPu6pRTrMWh+/ePeB0XpVTjELU89oBOFOHAHsjSdX7z0ZcssQL63Lme\nB+/d2wroF19sLRZtsx1KKVUfO4E9Lm+e2r6ReOCAyMKFIhdc4HlDFEQuvFBk0aLwt0MppWpg4+Zp\nXAb2oDNfDhwQmTdP5OyzvQf0/v1FliwJfzuUUqoOO4E9Lm+eBpz5Ul0Nb7xhDbksXeq+LSHBuhma\nnw89eoS3HUop5aCYCexOjkXXm/myfz/Mng1TpsCqVe5vTk6GQYOstMWsLFvnD7gdSikVDsF28e0+\n8DMU4/RYtK/jlX/3vcjf/iaSnu453NKsmcidd4qsW2frnJH4XEqpxgcbQzExkRUTjgJWrpkv3drt\n58Fu0Gbm8/DLL+47tmhhrSE6YgS0a2frXIG2o96KkF7ep9k0SjVuUSkC5oRwjEVnZqbz0uPDYdo0\neOxp2LbNfYfDD7eC+W23QatWts8TUDuCvDg5Xs9GKdWoxMRg76GxaFchjEX/8guMHWtN8S8qcg/q\nnTrBo4/CmjVWLnoYg7pdBQUzXYI6QHNKSydSUDAziq1SSjUUMRHYi4sHkZVVyKHgXru25qDgDlRR\nYQ2rZGRY9Vx2uazQ162bVVa3rMzqqTev+wshdmg2jVIqFDExFBNyAatVq6wMl5deslYuctW9u1XH\n5dprrZWL6hELY9uaTaOUCkmwd1vtPgjHBKWlS0WuuUbEGM8sl9NPF/nHP0SqqwM+XKxkscRKO5RS\n0UdDy4qx3TtevBj+8hd45x3PbX36WGPnF1xQbx2XumJpeTm72TRKqfjSoLJigs78EIEFC2DSJFi4\n0HP7JZdYAb13b9ttiqWxbS3Bq5SyK2qDtgFnfojA22/DWWdZvXDXoG4MXHUVfP01vPtuSEEdwpCd\no5RSURC1iFVv77i6Gl55BU46CS6/HD7//OBe1cbwfocTGXvprZQ/ONWqi+4Ax7JzlFIqiqI2FOMr\n86NLhwMwY4aV5fLjj27vkSZNePmw47mncjZrNnaHebt5Y+Wh4ZtQM1rqZue0aFGJSBKDBz+nsz+V\nUg1HsHdb7T6okxVTN/OjGZtkYtsc2d+xo2eGS/PmIqNHy21XjvJZBjdS9WY0M0UpFUlEqx47cDHw\nPbAaGOdjH48Gl5WtkT9dky/Tu50v25umeAb0Vq1ECgpEfv1VRESys+/1Wi49J+dex2ufay11pVQs\nsBPYQx6KMcYkAE8AfYENwBfGmLki8r3fN/76K5nPz+CZD56G7dvdt7VvD6NGwbBhVpGuGv4m7jid\n0RJLGTJKKRUMJ26engH8ICIVIrIfeAW4wufeGzbA6NFWHZfiYveg3qULPP64Vcdl3Di3oA7+b246\nndGiGTJKqYYq5AlKxpj/AS4SkVtqnucBZ4jIHXX2E/nzn+G552DfPveDHHWUNe0/Lw+aNPF7Pl8T\nd7zlxWdlBXZj1ds2wO/xlFIqEuxMUHIisF8FXFgnsPcUkTvr7Ceu022ygewTT4QJE6xc9MTEkNoB\n9oI++A7ggM7+VEpFVElJCSUlJQefT5w4MSqB/UygSEQurnmejzXY/0Cd/Q6dqVcva5boZZd5nfbv\ndCEuf6UCgJgpI6CUUnVFq6TAF8BRxph04GfgOuB6r3uef77VQz//fJ91XMKxyIS/G6HW1UZvkiql\n4kfIdwJFpBq4HfgAWAG8IiIrve780UfQt6/f4lzBLjJRXl5BXt5EcnIKycubSHl5hcc+/m6E6k1S\npVS8cWTmqYi8Bxxb3379+49EJInKyhSfQyzBpBkG2rsvLh7E4sWFHuPoxcXWOLq/bUop1dBEtKTA\n3LkGKMJfEA5mkQnfvXv38fH6FvIIaZEPpZSKMRGtxw67qO8mZX1pi65ycgopKZnoca6cnEIWLPB8\nXSmlGpoGUI+9/iGWYJbJ0yXklFLKU8z12IMRTO9eKaUaoqhMUAr4RMYIjASKcTII6xJySql4FvOB\n/YorRiCSxM6dKRqElVIqADEf2CN1LqWUihd2ArveZVRKqTijgV0ppeKMBnallIozGtiVUirOaGBX\nSqk4o4FdKaXijAZ2pZSKMxrYlVIqzmhgV0qpOKOBXSml4owGdqWUijMa2JVSKs5oYFdKqTgT4RWU\n7Kmtub5+/QGfi2ArpZSyxHzZXl0lSSnVmEW8bK8x5kFjzEpjzFJjzOvGmBahHM+bgoKZLkEdoDml\npRMpKJjp9KmUUiouhDrG/gHQXUROBn4AxofeJHfr1x8gkEWwG5KSkpJoNyFm6HdxiH4Xh+h3EZqQ\nAruIzBeR2gi7GDgy9Ca569w5Adhd59XddOrUcO/76n+0h+h3cYh+F4fodxEaJ6PjYOCfDh4PgOLi\nQWRlFXIouFtj7MXFg5w+lVJKxYV6s2KMMR8CHVxfAgS4W0TertnnbmC/iLzsdAMzM9P58MPhFBRM\nZcOGAzWLYOuNU6WU8iXkrBhjzB+BW4DzRWSvn/10JWullLIh2KyYkPLYjTEXA2OB8/wFdTsNU0op\nZU9IPXZjzA9AE2BLzUuLReRWJxqmlFLKnohNUFJKKRUZYc8ZNMZcbIz53hiz2hgzLtzni1XGmCON\nMQuMMd8ZY741xtwR7TZFmzEmwRjztTHmrWi3JZqMMS2NMa/VTPZbYYzpFe02RYsxZqQxZrkxZpkx\nZrYxpkm02xRJxpgZxpiNxphlLq+1NsZ8YIxZZYx53xjTsr7jhDWwG2MSgCeAi4DuwPXGmOPCec4Y\nVgWMEpETgLOA2xrxd1HrTuC7aDciBjwGvCsixwMnASuj3J6oMMZ0AoYDp4rIiVj3AK+Lbqsi7nms\neOkqH5gvIscCCwhgImi4e+xnAD+ISIWI7AdeAa4I8zljkoj8IiJLa/69C+v/vJ2j26roMcYcCfwB\neDbabYkmY0wacK6IPA8gIlUiUhnlZkVTItDcGJMEpAAbotyeiBKRT4FtdV6+Anih5t8vAP3rO064\nA3tnYJ3L859oxMGsljEmAzgZ+Dy6LYmqR4G7sOZENGbdgF+NMc/XDEtNN8Y0i3ajokFENgAPA2uB\n9cB2EZkf3VbFhPYishGsDiLQrr43hDuwe0txbNT/RzbGpAJzgDtreu6NjjHmUmBjzS8Yg/f/ThqL\nJOBU4G8icirwG9ZP70bHGNMKq3eaDnQCUo0xN0S3VQ1TuAP7T0BXl+dH0sh+Wrmq+Xk5B3hRROZG\nuz1R1Bu43BhTBvwfkGOMmRXlNkXLT8A6Efmy5vkcrEDfGF0AlInIVhGpBt4Azo5ym2LBRmNMBwBj\nzBHApvreEO7A/gVwlDEmvebu9nVAY86AeA74TkQei3ZDoklEJohIVxHphvXfxAIRuTHa7YqGmp/Y\n64wxx9S81JfGe0N5LXCmMeYwY4zB+i4a443kur9i3wIG1fz7j0C9ncKwrqAkItXGmNuxyvsmADNE\npDH+oTDG9AZygW+NMd9gDUlNEJH3otsyFQPuAGYbY5KBMuCmKLcnKkRkiTFmDvANsL/mf6dHt1WR\nZYx5GcgG2hpj1gKFwBTgNWPMYKyL39X1HkcnKCmlVHxpuEXNlVJKeaWBXSml4owGdqWUijMa2JVS\nKs5oYFdKqTijgV0ppeKMBnallIozGtiVUirO/D849bOHcCmSpgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f1a4cff5128>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot the results\n",
    "plt.plot(x_vals, y_vals, 'o', label='Data')\n",
    "plt.plot(x_vals, best_fit, 'r-', label='Best fit line', linewidth=3)\n",
    "plt.legend(loc='upper left')\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
